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DETERMINING  LITERACY  REQUIREMENTS  OF  JOBS: 
PROGRESS  AND  PROSPECTS  FOR  PROJECT  REALISTIC 


Thomas  G.  Sticht,  Richard  P.  Kem> 

John  S.  Caylor,  and  Lynn  C.  Fox 

Work  Unit  REALISTIC  was  initiated  by  the  Htman  Resources  Research 
Organization  in  April  1968  at  the  request  of  the  Office  of  the  Assistant 
Secretary  of  Defense  (Mai^ower)  and  the  Deputy  Chief  of  Staff  for 
Personnel,  Department  of  the  AntQr.  The  requests  were  made  because  of 
awareness  of  the  low  reading  skills  that  characterize  men  being  admitted 
into  the  military  services,  the  Army  in  particular,  under  Project  100,000. 
There  was  concern  that  a  number  of  the  "new  standards"  men  might  be 
assigned  to  jobs  wherein  the  demands  for  reading  skills  might  far  exceed 
their  ability  and  hence  lead  to  job  failure.  To  aid  in  decision  making 
for  dealing  with  this  possibility,  information  was  needed  concerning 
the  literacy  demands  of  various  Military  Occupational  Specialties  (MOSs). 

The  general  objectives  of  Work  Unit  REALISTIC  are  to  provide 
(1)  information  on  the  literacy  requirements  of  jobs  in  several  major 
MOSs,  and  (2)  information  and  suggestions  for  reducing  discrepancies 
between  persotmel  literacy  skill  levels  and  the  literacy  skill  levels 
required  by  the  job. 

In  pursuing  these  objectives,  one  of  our  first  problems  was  to 
define  "literacy"  skill.  Obviously,  this  refers  to  reading  ability, 
but  what  is  not  so  obvious  is  the  definition  of  "reading  ability." 

Most  e]q>erts  in  the  field  recognize  that  reading  is  a  conplex  of 
skills  and  abilities.  It  is  also  generally  recognized  that  reading 
involves  the  ability  to  translate  printed  symbols  into  speech  or  audi¬ 
tory  symbols,  and  that,  in  the  early  stages  of  reading  training,  most 
readers  silently  "speak"  the  words  on  the  printed  page  to  themselves. 

With  advancement  in  reading  skill,  there  appears  to  be  less  need  to 
sub-vocalize  the  words,  resulting  in  a  more  direct  decoding  of  printed 
symbols  into  meaning. 

Of  particular  concern  to  us  was  the  fact  that  listening  ability 
typically  precedes  reading  ability,  and,  in  fact,  the  reading  process 
is  usually  built  on  a  structure  of  listening  ability.  Because  of  this, 
we  conjectured  that  some  of  the  low  aptitude  men  in  Project  100,000 
might  not  be  able  to  learn  a  job  well  by  reading,  but  would  be  able 
to  learn  by  listening  or  "show-and-tell."  Their  primary  literacy 
defect  might  be  one  of  lack  of  skill  in  decoding  printed  symbolism. 

Thus,  we  decided  to  include  listening  as  a  component  of  literacy.  At 
the  request  of  the  sponsors,  we  have  also  considered  arithmetic  ability 
as  a  component  of  literacy  skill. 

Having  decided  on  the  skill  components  to  be  included  under  the 
rubric  of  "literacy,"  the  next  step  was  to  determine  how  to  find  out 
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what  the  requirements  for  these  skills  are  in  different  Army  MOSs. 

To  accomplish  this,  we  formulated  two  approaches  used  in  sub-units, 
REALISTIC  I  and  REALISTIC  li. 

The  approach  taken  in  REALISTIC  I  to  study  the  literacy  require¬ 
ments  of  'arious  Army  job  duties  consists  of  comparing  the  literacy  . 
skills  of  o.en  in  selected  MOSs  to  their  performance  on  job  proficiency 
tests.  By  the  means  shown  in  Figure  1,  we  hope  to  find  out  to  what 
extent  literacy  is  related  to  job  proficiency. 

REALISTIC  I — Research  Approach 

LITERACY  (or.  JOB  SAMPLE 

SKILLS  PROFICIENCY  MOSs 

Reading  Job  Sample  Armor  11 E 

Listening  Job  Knowledge  Mechanic  63C 

Arithmetic  Supervisor  Ratings  Supply  76Y 

Cook  94B 

CONTROLS:  Armed  Force:  Qualifications  Test  Scores 
and  Time-On-The-Job 

Figure  1 

The  literacy  tests  being  used  measure  reading,  listening,  and 
arithmetic  skills.  The  reading  and  arithmetic  tests  were  selected 
from  several  commercially  available  standardized  tests.  The  listening 
test  was  constructed  under  REALISTIC.  It  consists  of  three  brief 
selections  about  military  activities  presented  on  a  tape  recording. 
Questions  are  asked,  orally,  after  each  selection,  and  the  men  write 
their  responsv^s  on  answer  sheets.  The  test  requires  about  20  to  30 
minutes  to  administer,  and  is  rapidly  scored  by  hand. 

The  job  proficiency  tests  consist  of  job  sample  tests,  which  are 
"hands-on"  performance  tests,  paper- and-pencil  job  knowledge  tests, 
and  supervisor  ratings.  These  tests  were  constructed  and  administered 
in  HumRRO  Work  Unit  UTILITY.  Literacy  and  job  performance  data  were 
obtained  for  men  in  five  combat  and  combat- support  MOSs  into  which 
many  of  the  men  of  lower  literacy  skill  levels  are  assigned.  The  MOSs 
surveyed  include  the  Armor  Crewman  (HE),  Mechanic  (63C),  Supply 
Clerk  (76Y),  Cook  (94B],  and  Medical  Coipsman  (91A).  The  Medical 
Corpsman's  MOS  will  be  emitted  frexa  this  paper  because  initial  data 
analyses  have  not  yet  been  completed. 

In  Work  Unit  UTILITY,^  job  performance  tests  were  constructed  so 
as  to  include  tasks  representative— in  terms  of  essentiality,  frequency, 
and  urgency— of  tasks  performed  by  men  in  the  five  MOSs.  Thbrefore, 


^Robert  Vineberg,  "Progress  Report  on  Work  Unit  UTILITY,"  HumRRO 
Professional  Paper  6-70,  March  1970. 
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an  examination  of  the  relationships  between  literacy  test  performance 
and  performance  on  the  job  sample  tests  indicates  the  extent  to  which 
different  levels  of  reading,  listening,  and  arithmetic  skills  may  have 
prcmoted,  in  a  general  manner,  better  job  performance. 

At  the  present  time,  all  data  collection  is  complete  and  data 
analysis  is  under  way.  In  this  paper,  scsne  reading,  listening,  and 
job  performance  data  from  our  first  level  of  analysis  will  be  presented. 
These  are  data  for  which  many  sub-analyses  will  be  made  in  the  future, 
including  analyses  of  the  subtests  of  the  job  performance  test  batteries 
in  the  various  MOSs. 

A  summary  of  some  of  the  data  obtained  with  Non-Mental  Category  IV^ 
personnel.  Figure  2  shows  the  relationships  between  reading  and  listening 
ability  on  the  horizontal  axis.  Percent  correct  on  the  job  performance 

Scores  of  Non-Monfol  Cotgory  IV  Porsonnol 


RoodUig  Grod*  L«r«l  UMsning  Sear* 

Figure  2 

tests  for  four  MOSs  are  shown  on  the  vertical  axis.  As  might  be 
expected,  there  is  considerable  variability  among  the  MOSs,  particularly 
with  regard  to  reading  and  job  performance.  It  also  appears,  however, 
that  for  both  reading  and  listening,  the  Armor  Crewmen  and  Mechanics 
(the  top  two  curves)  tend  to  group  together,  and  the  Cook  and  Supply 
MOSs  (bottom  two  curves]  tend  to  group  together,  suggesting  that  the 
mechanically  oriented  MOSs  may  differ  in  their  literacy  skill  require¬ 
ments  from  the  non-mechanical  MOSs. 

Another  general  feature  to  be  noted  is  the  change  in  job  performance 
as  reading  and  listening  skill  level  increases.  The  general  trend  can 
be  traced  by  the  solid  lines,  labeled  H.  These  are  straight  lines 

^Armed  Forces  Qualification  Test  (AFQT)  Groiq>:  Category  I  (93-99); 
Category  II  (65-92);  Category  III  (31-64);  Category  IV  (10-15). 
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fitted  to  the  average  job  performance  scores  for  the  four  MOSs  coiid>lned. 
If  we  ignore  the  bend  in  the  reading  curve  and  note  the  change  in  job 
performance  scores  with  reading  scores  increasing  from  fifth  grade  to 
13  plus,  the  job  performance  increase  amounts  to  about  14%.  This  con¬ 
trasts  with  the  22%  increase  in  average  job  performance  that  occurs 
over  the  range  of  listening  scores  from  10  to  35.  These  data  support 
our  original  conjecture  that  listening  skill  may  be  equally  or  more 
important  than  reading  skill  for  successful  job  performance.  Additional 
support  for  this  idea  is  provided  in  Figure  3. 

Scores  of  Mental  Category  IV  Personnel 


Reading  Grade  Level  Listening  Score 

Figure  3 

The  same  types  of  information  are  presented  in  Figtjres  2  and  3,  but 
the  latter  deals  with  the  lower  aptitude.  Mental  Category  IV  men.  Again 
there  is  a  tendency  for  the  Armor  and  Mechanic's  data  to  form  one  group, 
and  the  Cook's  and  Supply  data  to  form  a  second  group.  This  is  more 
evident  for  reading  than  listening.  The  bend  in  the  reading  curve 
occurs  at  grade  6  for  the  Non  IVs,  and  grade  5  for  the  IVs,  although 
further  analyses  are  needed  before  we  understand  the  significance  of 
this  bend  in  either  mental  category  group. 

It  is  again  observed  in  Figure  3  that  there  is  a  fairly  strong 
relationship  between  listening  and  job  performance.  This  is  even  more 
interesting  in  the  case  of  the  Category  IV  personnel  because  they 
represent  a  fairly  homogeneous  aptitude  group  on  AFQT,  yet  the  listen¬ 
ing  test  appears  to  be  performing  an  additional  sorting-out  of  personnel 
within  this  aptitude  grouping  into  poorer  and  better  job  performance. 

Analyses  similar  to  these  shown  in  Figures  2  and  3  are  being  per¬ 
formed  with  the  arithmetic  data,  with  job  knowledge  test  scores,  and 
with  supervisor  ratings  as  the  indices  of  job  proficiency.  In  the 
future,  we  have  to  analyze  the  job  sample  performance  data  with  regard 
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to  the  number  of  pronqpts  required  by  men  of  different  literacy  levels, 
the  use  of  manuals  by  men  of  different  literacy  levels,  and  so  forth. 

Let  me  say  that  from  the  combined  results  of  our  analyses  of  the 
REALISTIC  I  data,  we  hope  to  obtain  extensive  information  bearing 
on  the  question  of  the  literacy  requirements  for  these  MOSs. 

However,  REALISTIC  I  provides  only  partial  information  concerning 
the  literacy  requirements  for  the  representative  MOS  because  literacy 
tasks  per  se  were  not  included  in  the  job  performance  test  battery 
developed  by  Work  Unit  UTILITY,  from  which  REALISTIC  I  job  proficiency 
data  were  obtained.  In  fact,  literacy  tasks  per  se  have  not  been 
clearly  identified  in  previous  job  or  task  analyses  for  an/  MOS.  For 
this  reason,  we  formulated  our  second  approach  to  the  evaluation  of 
literacy  requirements  of  major  MOSs  -  Sub-Unit  REALISTIC  II. 

The  first  general  objective  of  REALISTIC  II  is  to  identify  reading, 
listening,  and  arithmetic  tasks  in  a  set  of  MOSs  representative  of 
mechanical,  clerical,  and  service  occupations.  For  this  purpose,  we 
have  selected  three  of  the  same  MOSs  examined  in  REALISTIC  I,  Mechanic 
(MOS  63C),  Supply  (MOS  76Y),  and  Cook  (MOS  94B).  These  MOSs  offered 
the  possibility  of  extending  some  of  the  UTILITY  and  REALISTIC  I  data 
to  REALISTIC  II.  At  Fort  Ord  and  Fort  Carson  we  obtained  information 
from  some  of  the  same  men  tested  in  UTILITY  and  REALISTIC  I. 

Our  second  objective  in  REALISTIC  II  is  to  identify  reading  skill 
levels  sufficient  for  effective  performance  of  job-related  reading 
tasks.  The  third  objective  is  to  suggest  methods  for  modifying  reading 
tasks  to  reduce  the  need  for  literacy  skill  in  certain  jobs.  This 
will  include  recommendations  for  the  redesign  of  reading  materials, 
and  the  substitution  of  listening  for  reading  tasks  where  this  will 
facilitate  job  or  training  performance. 

To  identify  job-related  reading  tasks,  we  developed  a  structured 
interview  that  was  administered  to  men  in  the  target  MOSs  at  their  job 
locations.  Men  were  selected  for  interview  on  the  basis  of  AFQT,  time- 
on-the-job,  and  reading  grade  level  of  ability  as  determined  by  pretests. 
An  effort  was  made  to  include  as  many  men  as  possible  who  had  been 
tested  in  UTILITY  and  REALISTIC  I  research. 

The  information  obtained  with  the  structured  interview  included 
personal  data,  and  a  description  of  a  "typical"  work  day,  in  which  a 
man  attenqjted  to  describe  various  activities  he  might  perform.  There 
were  also  three  questions  designed  to  obtain  information  about  the 
listening,  reading,  and  arithmetic  tasks  that  a  man  might  perform  on 
the  job.  For  listening  tasks,  the  man  was  asked  to  give  five  instances 
when  he  had  asked  somebody  for  job-related  information  in  the  last 
month  or  so.  Questioning  was  continued  until  the  man  said  he  could 
think  of  no  other  such  instances. 

/  similar  procedure  was  used  for  identifying  reading  tasks.  The 
man  was  asked  to  give  examples  of  when  he  had  been  doing  some  job 
activity  and  had  used  printed  materials — a  manual,  a  form,  a  letter, 
and  so  forth.  Usually  this  brought  a  report  of  the  use  of  some  manual. 
The  man  was  then  asked  to  get  the  manual  and  locate  the  exact  page  he 
had  used.  This  came  as  a  surprise  to  many  of  them,  and  we  observed 
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Infornally  many  instances  where  a  m  n  took  IS  to  20  minutes  to  locate  a 
page,  aid  sometimes  failed  to  .'cate  it  at  all.  This  failure  appeared 
to  result.,  at  times,  from  the  difficult  indexing  system  of  some  of  the 
manuals,  and  at  other  times,  we  found  that  the  man  had  not  actually 
used  the  manual  before,  having  depended  on  one  of  his  buddies  to 
locate  the  section  he  needed.  After  obtaining  as  many  citations  of 
printed  materials  as  possible,  up  to  a  maximum  of  five— which  most 
men  were  unable  to  report — we  asked  for  five  examples  of  the  use  of 
arithmetic.  We  found  very  little  use  of  arithmetic. 

We  also  solicited  ways  to  modify  materials  to  make  than  easier  to 
use,  but  most  men  reported  them  to  be  satisfactory,  even  though  they 
often  couldn't  use  the  materials  they  brought  to  us. 

From  the  interviews  we  obtained  a  great  deal  of  information  to 
help  identify  literacy  tasks  in  these  MOSs.  One  line  of  analysis  that 
we  have  accomplished  is  shown  in  Figure  4.  The  figure  shows  the  average 
reading  grade  level  of  difficulty  of  publications  in  five  MOSs  as 
estimated  by  the  Flesch  readability  formula.  The  figure  also  shows 
the  reading  abilities  of  men  in  these  MOSs,  differentiating  for  Non- 
Category  IV  for  Category  IV  men.  For  our  purposes,  the  bars  of  interest 
are  those  concerning  the  Supply,  Mechanic,  and  Cook  MOSs.  There  are 
large  discrepancies  between  the  readability  of  the  manuals  and  the 


'  '  Readability  of  Publications  in  Use  in  Various  MOSs 


AbUity  of  Noo  IV 
ftootflnf  Ability  of  Cot  )V 
Roodobllfty  of  Pvbllcotlono 


reading  abilities  of  the  men  in  the  MOSs.  This  discrepancy  is  less 
for  Cooks,  because  their  materials  are  less  difficult.  One  of  the 
questions  of  concern  is  whether  the  differences  between  readability 
(i.e.,  ease  of  understanding  the  materials)  and  reading  ability  levels, 
make  any  real  difference  on  how  much  people  in  the  MOS  will  use 
the  manuals. 

In  Figure  5  data  relevant  to  this  question  are  presented.  On  the 
left,  we  have  replotted  the  readability  figures  for  publications  in  the 
three  MOSs — 76Y,  Supply;  63C,  Mechanics;  and  94B,  Cooks.  On  the  right 
we  have  plotted  data  obtained  with  otir  interview.  These  are  "readership" 


Readability  and  Readership  in  Three  MOSs 
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Figure  5 

data,  referring  to  the  extent  to  which  men  in  the  MOS  reported  the  use 
of  printed  materials — the  higher  the  bar,  the  greater  the  extent  of  use. 
Of  interest  is  the  fact  that  the  readership  is  low  in  the  Supply  MOS, 
where  reading  difficulty  of  materials  is  high,  and  the  readership 
increcses  in  those  MOSs  where  the  readability  of  the  reading  materials 
improves.  Thus,  as  the  reading  difficulty  decreases,  the  readership 
increases. 

Figure  6  shows  the  same  data  as  Figure  5,  but  in  this  case,  the 
readability  of  materials  is  plotted  on  the  horizontal  axis,  and  the 
readership  on  the  vertical  axis.  In  addition,  there  are  data  for  men 
of  three  reading  ability  levels,  who  scored  at  grades  4  to  6.9,  7  to 
8.9,  or  9th  grade  or  beyond,  on  our  reading  tests.  It  is  shown  that 
the  Cooks  (94B),  who  have  the  easiest  materials,  also  show  the  greatest 
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use  of  publications.  The  Mechanic  MOS  materials  are  more  difficult, 
and  the  readership  index  in  this  MOS  declines.  The  Supply  MOS  publica¬ 
tions  are  the  most  difficult,  and  they  have  the  lowest  readership 
index.  The  readership  score  for  the  higher  reading  ability  groups 
does  not  decrease  as  much  as  that  for  the  lower  ability  readers, 
suggesting  that  the  more  difficult  the  reading  materials  are,  the 
more  the  poor  readers  tend  to  avoid  them.  In  fact,  they  may  tend  to 
rely  more  on  listening  sources  than  reading  sources  for  job  information, 
as  is  suggested  in  Figure  7. 

We  have  plotted  an  information  modality  index  in  Figure  7.  This 
provides  an  indication  of  the  extent  to  which  men  in  an  MOS  tend  to 
rely  more  on  listening  than  on  reading  for  getting  job-related  informa¬ 
tion.  In  Figur  7,  scores  below  100  mean  that  the  different  reading 
ability  groups  tended  to  rely  more  on  listening  for  job  information, 
and  scores  above  100  mean  that  reading  materials  were  used  more  than 
listening  sources  for  obtaining  job-related  information. 

The  poorer  readers  in  the  Supply  and  Mechanic  MOSs  tend  to  rely 
mere  on  listening  than  reading,  as  is  shown,  whereas  in  the  Cook  MOS 
t.iis  is  not  true.  As  we  have  stated,  the  Supply  MOS  has  the  most 
difficult  printed  materials,  the  Mechanic  the  next  most  difficult, 
and  the  Cook  the  least  difficult  material.  Hence,  the  tendency  to 
use  listening  sources  is  related  to  both  reading  ability  of  the  men 
and  reading  difficulty  of  publications  in  these  MOSs. 

While  the  import  of  these  data  will  not  be  clearly  evident  until 
further  analysis  of  the  MOS  printed  materials,  early  data  in  Work  Unit 
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Figure  7 

UTILITY  have  indicated  that  mechanics  who  used  manuals  in  performing 
their  job  sample  tests  tended  to  do  better  than  men  who  did  not  use 
the  manuals.  This  was  true  for  both  Category  IV  and  Non-Category  IV 
men.  Thus,  if  there  is  a  desire  to  have  men  use  more  printed  materials 
on  the  job,  perhaps  to  ensure  better  performance,  then  the  present  data 
on  readability,  reading  ability,  and  readership  across  the  three  MOSs 
suggest  that  this  might  be  accomplished  both  by  improving  literacy 
skills  of  the  men,  and  by  the  redesign  of  reading  materials.  Further¬ 
more,  the  present  data  suggest  that  greater  gains  in  readership  are  to 
be  expected  from  redesigning  materials  than  from  increasing  the  literacy 
skills  of  men.  Obviously,  there  are  implications  for  selection,  classi¬ 
fication,  and  training  in  the  data. 

At  the  present  time  we  are  obtaining  copies  of  all  printed  materials 
used  in  the  on-the-job  interviews  and  classifying  them  into  six  content 
categories  referring  to  the  type  of  information  presented:  Standards 
and  Specifications,  Identification  and  Physical  Description,  Procedural 
Directions,  Functional  Descriptions,  Procedural  Check  Points,  and 
Theory.  "Standards  and  Specifications"  include  such  things  as  gap  and 
point  settings,  and  type  of  form.  The  category  "Procedural  Directions" 
might  give  procedures  for  assembling  a  piece  of  equipment,  removing  a 
part,  baking  a  cake,  and  so  forth.  We  are  also  classifying,  according 
to  these  same  content  types,  the  kind  of  information  the  men  said  they 
were  trying  to  find  in  the  printed  materials.  We  have  found,  for 
instance,  that  a  man  was  seeking  Standards  and  Specifications  in 
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material  that  offers  Procedural  Directions — or  he  may  have  been  using 
material  that  presents  Standards  and  Specifications  to  obtain  an  identi¬ 
fication  of  a  part.  We  are  attempting  to  use  the  various  classification 
systems  to  define  reading  tasks  in  terms  of  the  type  of  information  a 
man  seeks  from  a  given  content  type  of  reading  material. 

We  are  also  classifying  the  different  content  type  materials  accord¬ 
ing  to  their  format;  for  example,  whether  the  material  is  in  tabular  or 
narrative  format.  Obviously,  the  reading  task  of  seeking  Standards  and 
Specifications  in  a  Procedural  Directions  type  of  content,  presented  in 
narrative  format,  is  a  much  different  reading  task  than  seeking  standards 
and  specifications  in  a  Standards  and  Specifications  type  of  content, 
presented  in  tabular  format. 

As  soon  as  the  identification  of  reading  tasks  has  been  completed, 
we  intend  to  construct  reading  task  performance  tests,  and  determine  how 
well  men  of  different  reading  ability  levels  can  perform  the  different 
reading  tasks.  Among  other  things,  this  information  will  Indicate  the 
reading  skills  needed  to  successfully  perform  job-related  reading  tasks. 

Research  under  REALISTIC  has  also  involved  a  series  of  experimental 
studies  exploring  reading  and  listening  skills  for  men  of  different 
aptitude  levels  (REALISTIC  III).  Emphasis  has  been  on  the  experimental 
evaluation  of  the  use  of  time-compressed  speech  as  an  audio  job  or 
training  performance  aid.  Some  of  the  major  conclusions  in  a  tech¬ 
nical  report  completed  under  REALISTIC  III,  are  that,  where  feasible, 
reading  and  listening  materials  should  be  available  to  students,  so  that 
those  who  learn  better  with  one  modality  than  the  other  may  exercise 
that  skill.  Also,  listening  materials  can  be  subjected  to  acceleration 
in  speech  rate  to  produce  rates  around  250  to  275  words  per  minute 
without  seriously  impairing  comprehension.  This  is  true  for  both  high 
and  low  aptitude  men,  and  thus  permits  listening  rates  comparable  to 
average  silent  reading  rates. 
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